Field of the Invention
The present invention relates generally to an image processing technology for improving the quality of an image generated by image pickup, and more particularly to an image processing technology for generating an image in which a deterioration caused by an aberration is restrained using a plurality of parallax images generated by the image pickup.
Description of the Related Art
A captured image generated by an image pickup apparatus, such as a digital camera, has a blur (deteriorated image quality) caused by a micro spread in light emitted from one point under the influence of an aberration, a diffraction, etc. of an image pickup optical system. A point spread function (“PSF”) is a function having the micro spread of the light.
It is a known technology to generate a plurality of parallax images having different parallaxes by capturing a plurality of light fluxes that have passed different areas in an exit pupil an image pickup optical system, with different photoelectric converters (pixels or subpixels). However, there is a difference other than the parallax component derived from the different PSF in the plurality of parallax images, since an aberration generally depends on an area in which the light flux passes among the exit pupil.
Japanese Patent Laid-Open No. (“JP”) 2013-236207 discloses an image processing method for preparing a filtering function based on the PSF for each of the plurality of parallax images and for correcting a deterioration of each parallax image by performing filtering processing using the filtering function corresponding to each parallax image. JP 2013-179564 discloses an image processing method for reducing ghost (unnecessary light) based on a difference of a plurality of parallax images.
However, the image processing method disclosed in JP 2013-236207 needs plural pieces of PSF data used for and to generate a filtering function by the number of parallax images. Since the PSF changes according to an image pickup condition, such as a focal length and an aperture value of an image pickup optical system, and a position on an image, data are necessary by the number of image pickup conditions and positions. The filtering processing in JP 2013-236207 is convolution processing of a filter (kernel) of a size (dozens taps times dozens taps) larger than a spread of the PSF, and the processing load is heavy. A necessary data amount and processing load increase as the number of parallax images increases. However, it is difficult to correct the deterioration caused by the aberration because pixel value information lacks in an area (pixel value saturated area) in which a pixel value is saturated in a captured image.
The image processing method disclosed in JP 2013-179564 is based on the premise that the pixel value increases by the ghost, and cannot obtain a sufficient deterioration restraining (aberration correcting) effect when the pixel value increases and decreases for each pixel and the aberration occurs.